Method and system for generating an output with respect to a group of individuals

ABSTRACT

There is provided a computer-implemented method for generating an output with respect to a group of individuals. The method includes: identifying the group of individuals amongst a plurality of individuals in an area being monitored by one or more sensors; determining, for each individual in the group of individuals, one or more individual-based features associated with the individual based on sensing data obtained from the one or more sensors; determining a group characteristic information associated with the group of individuals based on the one or more individual-based features determined for each individual in the group; and generating the output based on the group characteristic information determined for the group of individuals. There is also provide a corresponding system for generating an output with respect to a group of individuals.

TECHNICAL FIELD

The present invention generally relates to a computer-implemented methodfor generating an output with respect to a group of individuals and asystem thereof, such as but not limited to, targeted information (e.g.,a targeted advertisement content).

BACKGROUND

In a wide variety of applications, it may be desirable to generate anoutput (e.g., provide or present targeted information, such as atargeted advertisement content) with respect to various individualsdetected by a sensor. For example, a conventional system may beconfigured to determine a feature, such as a characteristic or anattribute, associated with an individual and then output a targetedadvertisement content based on the feature of the individual determined.For example, if the individual is detected to be young, then a targetedadvertisement content targeting young people may be provided. On theother hand, if the individual is detected to be old, then a targetedadvertisement content targeting elder people may be provided instead.

However, such conventional systems of providing targeted information mayonly be capable of targeting information directly in accordance with oneor more basic types of features detected, such as gender, age, emotionalstate, and so on, of a single individual, and thus, the categories ofinformation which may be targeted may be undesirably limited orrestricted. For example, conventional systems may only be able toprovide targeted information to an individual directly in accordance tothe individual's gender, age, and/or emotional state detected.

A need therefore exists to provide a method for generating an output,such as but not limited to, targeted information (e.g., a targetedadvertisement content), and a system thereof, that seek to overcome, orat least ameliorate, one or more of the deficiencies of conventionalmethods and systems, such as the conventional methods and systems asmentioned above. It is against this background that the presentinvention has been developed.

SUMMARY

According to a first aspect of the present invention, there is provideda computer-implemented method for generating an output with respect to agroup of individuals, the method comprising:

identifying the group of individuals amongst a plurality of individualsin an area being monitored by one or more sensors;

determining, for each individual in the group of individuals, one ormore individual-based features associated with the individual based onsensing data obtained from the one or more sensors;

determining a group characteristic information associated with the groupof individuals based on the one or more individual-based featuresdetermined for each individual in the group; and

generating the output based on the group characteristic informationdetermined for the group of individuals.

According to a second aspect of the present invention, there is provideda system for generating an output with respect to a group ofindividuals, the system comprising:

a memory;

at least one processor communicatively coupled to the memory andconfigured to:

-   -   identify the group of individuals amongst a plurality of        individuals in an area being monitored by one or more sensors;    -   determine, for each individual in the group of individuals, one        or more individual-based features associated with the individual        based on sensing data obtained from the one or more sensors;    -   determine a group characteristic information associated with the        group of individuals based on the one or more individual-based        features determined for each individual in the group; and    -   generate the output based on the group characteristic        information determined for the group of individuals.

According to a third aspect of the present invention, there is provideda computer program product, embodied in one or more non-transitorycomputer-readable storage mediums, comprising instructions executable byat least one processor to perform a method for generating an output withrespect to a group of individuals, the method comprising:

identifying the group of individuals amongst a plurality of individualsin an area being monitored by one or more sensors;

determining, for each individual in the group of individuals, one ormore individual-based features associated with the individual based onsensing data obtained from the one or more sensors;

determining a group characteristic information associated with the groupof individuals based on the one or more individual-based featuresdetermined for each individual in the group; and

generating the output based on the group characteristic informationdetermined for the group of individuals.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be better understood andreadily apparent to one of ordinary skill in the art from the followingwritten description, by way of examples only, and in conjunction withthe drawings, in which:

FIG. 1 depicts a schematic flow diagram of a method for generating anoutput with respect to a group of individuals according to variousembodiments of the present invention;

FIG. 2 depicts a schematic block diagram of a system for generating anoutput with respect to a group of individuals according to variousembodiments of the present invention;

FIG. 3 depicts a schematic drawing of an example computer system inwhich the system as shown in FIG. 2 may be implemented;

FIG. 4 depicts a schematic drawing illustrating an example overview of amethod for generating an output with respect to a group of individualsaccording to various example embodiments of the present invention;

FIG. 5 depicts a schematic drawing of an example skeleton model orrepresentation generated for an individual at two example posturesaccording to various example embodiments of the present invention;

FIG. 6 depicts a schematic drawing of three individuals along with theirrespective orientation shown according to various example embodiments ofthe present invention;

FIG. 7 depicts a schematic drawing of a top view of a plurality ofindividuals located in an area being monitored, along with theirrespective orientation shown, according to various example embodimentsof the present invention;

FIG. 8 depicts a schematic drawing of face models or representationsgenerated for three individuals according to various example embodimentsof the present invention;

FIG. 9 depicts a block diagram illustrate an example technique ofdetermining a group classification information associated with a groupof individuals according to various example embodiments of the presentinvention;

FIGS. 10A to 10C depict three schematic block diagrams of three examplecases, respectively, illustrating example types of features ofindividuals in a group and the example group characteristic informationthat may be determined for the group based on a classification of suchexample types of features, according to various example embodiments ofthe present invention;

FIG. 11 depicts an example group classification technique configured tooutput a group classification information based on various example typesof features of the individuals in the group, according to variousexample embodiments of the present invention; and

FIG. 12 depicts a block diagram illustrating an example system forgenerating an output with respect to a group of individuals according tovarious example embodiments of the present invention.

DETAILED DESCRIPTION

As described in the background, in a wide variety of applications, itmay be desirable to generate an output (e.g., provide or presenttargeted information, such as a targeted advertisement content) withrespect to various individuals detected by a sensor. For example, aconventional system may be configured to determine a feature, such as acharacteristic or an attribute, associated with an individual and thenoutput a targeted advertisement content based on the feature of theindividual determined. However, such conventional systems of providingtargeted information may only be capable of targeting informationdirectly in accordance with one or more basic types of featuresdetected, such as gender, age, emotional state, and so on, of a singleindividual, and thus, the categories of information which may betargeted may be undesirably limited or restricted.

For example, conventional approaches in “smart” advertisement may focussolely on individual-based features, such as age group (e.g., targetedadvertisement based on the age of an individual), gender (e.g., targetedadvertisement based on the gender of the individual, for example, maleand female may not be interested in the same items), clothing (e.g.,targeted advertisement based on the attire or style of an individual,for example, in the interest of recommending related shops or brands),emotional state (e.g., targeted advertisement based on the mood of anindividual). However, such conventional approaches are at an individuallevel only in the sense that they are only capable of providing targetedinformation directly in accordance with one or more basic types offeatures detected of a single individual. For example, it has beenidentified according to various embodiments of the present inventionthat no information about the context associated with the individual isobtained in such conventional approaches, such as, whether theindividual is present with his/her family as a group, whether theindividual is present with his/her partner as a group, and so on.Therefore, such conventional approaches are not capable of providinginformation on multiple individuals considered as a group. As a result,such conventional approaches are not able to, for example, providetargeted advertisement content that are applicable to a group ofindividuals (i.e., at a group level).

Various embodiments of the present invention provide acomputer-implemented method for generating an output, and a systemthereof, that seek to overcome, or at least ameliorate, one or more ofthe deficiencies of conventional methods and systems, such as theconventional methods and systems as mentioned in the background. Inparticular, various embodiments of the present invention provide acomputer-implemented method for generating an output with respect to agroup (or a set) of individuals, and a system thereof.

FIG. 1 depicts a schematic flow diagram of a method 100(computer-implemented method) for generating an output with respect to agroup of individuals (which may also be referred to as subjects orpersons). The method 100 comprises a step 102 of identifying the groupof individuals amongst a plurality of individuals in an area beingmonitored by one or more sensors; a step 104 of determining, for eachindividual in the group of individuals, one or more individual-basedfeatures associated with the individual based on sensing data obtainedfrom the one or more sensors; a step 106 of determining a groupcharacteristic information associated with the group of individualsbased on the one or more individual-based features determined for eachindividual in the group; and a step 108 of generating the output basedon the group characteristic information determined for the group ofindividuals. In the context of various embodiments, it will beunderstood by a person skilled in the art that the term “individual” mayrefer to an individual person. It will also be understood by a personskilled in the art that the term “individual-based feature” (which mayalso be interchangeably referred to as “individual-associated feature”)may refer to any feature associated with an individual (i.e., at anindividual level or with respect to the individual), such as acharacteristic or an attribute, of the individual.

In relation to step 102, for example, one or more groups of individualsmay be identified in an area being monitored. That is, one or moreindividuals amongst the plurality of individuals may be determined tobelong (or relate) as one group (i.e., same group) and may thus beassigned to one particular group. In this manner, one or more separategroups of individuals, each comprising one or more individualsdetermined to belong to the respective group, may thus be identified inan area being monitored. In this regard, since the method 100 isconfigured to identify a group of individuals, the method 100 isadvantageously able to generate an output with respect to the group ofindividuals identified (i.e., at a group level), for example, instead ofmerely with respect to a single individual detected (i.e., at anindividual level only). As a result, for example, the method 100advantageously broadens the categories of information which may betargeted, thus enabling more appropriate/relevant or better suitedinformation to be provided or presented to the group of individuals, forexample, taking into account one or more features at a group levelinstead of at an individual level. For example, the categories ofinformation are no longer limited or restricted to one or more basictypes of features associated with a single individual detected, such asgender, age, emotional state, and so on, but are broadened to be able toaccommodate for characteristics (e.g., relationship, context and so on)associated with a group of individuals detected (group characteristicinformation), such as but not limited to, a couple, a family, a group ofcolleagues, and so on.

By way of examples only and without limitation, in the case of a couple(group characteristic information) being detected as a group, a targetedadvertisement for a romantic holiday for two or a romantic dinner fortwo may be provided, and in the case of a family (group characteristicinformation) being detected as a group, a targeted advertisement for afamily holiday with a stay in a villa or a trip to the zoo may beprovided. Such categorizations or classifications of information are notpossible with conventional techniques that only take into account one ormore features associated with one individual (i.e., at an individuallevel). For example, although a middle age man may be detected, suchconvention techniques are not able to determine whether such a middleage man is present as a couple or as a family. Therefore, according toembodiments of the present invention, categories of information whichmay be targeted are advantageously broadened, thus enabling moreappropriate/relevant or better suited information to be provided orpresented to the group of individuals.

In relation to step 104, for example, for a group of individuals thathas been identified, one or more features (individual-based features)associated with each individual in the group may then be determinedbased on sensing data obtained from the one or more sensors. It can beappreciated by a person skilled in the art that the presented inventionis not limited to any particular types of features associated with anindividual that can be determined, as long as the feature(s) associatedwith the individual may be used for, or may facilitate in, categorizingor classifying purposes with respect to the individual.

In various embodiments, the one or more individual-based featuresdetermined may preferably be selected from the group consisting offacial feature(s), body feature(s), spatial feature(s) (e.g.,position/location of an individual, such as coordinates), and speechfeature(s). It can be understood by a person skilled in the art thateach type of these individual-based features may be determined by acorresponding classifier or analysis module, such as a facial featureclassifier configured to determine one or more facial features, a bodyfeature classifier configured to determine one or more body features, aspatial feature classifier configured to determine one or more spatialfeatures, and a speech feature classifier configured to determine one ormore speech features, based on the sensing data obtained from one ormore sensors.

In various embodiments, the one or more sensors are selected from thegroup consisting of a motion sensing device, a range sensing device(e.g., a time-of-flight (ToF) sensing device), an image capturingdevice, and an audio capturing device.

In relation to step 106, based on the one or more individual-basedfeatures determined for each individual in the group, a groupcharacteristic information for the group may subsequently be determined.In the context of various embodiments, the group characteristicinformation indicates one or more characteristics determined for thegroup of individuals as a whole (i.e., at a group level). By way ofexamples only and without limitations, the group characteristicinformation may indicate that the group is a couple, a family, happy,active, and so on. It will be appreciated by a person skilled in the artthat the group characteristic information may indicate any number ofcharacteristics as appropriate or as desired, such as a young couple ora happy family. In various embodiments, the group characteristicinformation may indicate a relationship amongst the individuals in thegroup (which may also be referred to as a group relationshipinformation), for example, a couple or a family as mentionedhereinbefore. In this regard, it will be appreciated by a person skilledin the art that various classification techniques/algorithms may beapplied for producing a classification result/output based on one ormore inputs obtained/detected, and the present invention is limited toany particular type of classification technique/algorithm.

By way of examples only and without limitations, various classificationtechniques/algorithms include support vector machine (SVM),probabilistic neural network (PNN), and k-nearest neighbour (KNN), whichare known in the art and thus need not be described herein. That is, itcan be understood by a person skilled in the art that any appropriateclassification technique known in the art may be applied to produce aclassification result/output (group characteristic information) based onthe one or more individual-based features determined for each individualin the group. By way of example, SVMs are supervised learning modelswith associated learning algorithms that analyze data for classificationand regression analysis, and SMVs may build a model based on trainingdata (e.g., training sets of one or a combination of types of featuresand the corresponding desired group characteristic information) fordetermining (e.g., predicting) the classification result/output based oninputs received (e.g., individual-based features determined for eachindividual in the group).

In relation to step 108, for example, the group characteristicinformation determined for the group of individuals may then be used togenerate an output for various applications as desired or asappropriate. In various embodiments, the group characteristicinformation may be used to generate an output selected from the groupconsisting of a targeted advertisement content, a targeted securityalert (e.g., an illegal or suspicious gathering of a group ofindividuals), an environment setting (e.g., a light setting and/orbackground music setting), and a signal or message comprising the groupcharacteristic information. However, it will be appreciated to a personskilled in the art that the present invention is not limited to suchapplications and may be applied to generate an output for various otherapplications as desired or as approripate, as long as the output isdesired to be dependent or based on the group characteristic informationassociated with a group of individuals determined. For example, it willbe appreciated by a person skilled in the art that the above-mentionedoutput may be a signal or message comprising the group characteristicinformation for transmission to one or more devices or systemsconfigured for specific application(s) (e.g., for generatingadvertisement content, security alert or environment setting), such thatthe one or more devices or systems generate their output based on thesignal or message received. In other words, it will be appreciated by aperson skilled in the art that a device or system configured forspecific application(s) may be further configured to perform the method100 to generate an output in the form of a content, an alert or aneffect based on the group characteristic information (e.g., anadvertisement display system, such as a digital signal system, may beconfigured to produce/display an advertisement content based on thegroup characteristic information), or a device or system (e.g., may bereferred to as a group characteristic information device or system) maybe configured to perform the method 100 to generate an output in theform of a signal or message comprising the group characteristicinformation for transmission to one or more devices or systemsconfigured for specific application(s), such that the one or moredevices or systems generate their output based on the signal or messagereceived. For example, the group characteristic information device orsystem may output a signal or message comprising the groupcharacteristic information to an advertisement display system such thatthe advertisement display system generates/displays an advertisementcontent based on the group characteristic information in the signal ormessage received.

Accordingly, the method 100 according to various embodiments of thepresent invention advantageously broadens the categories of information(e.g., advertisement content or control signals (e.g., environmentsettings)) which may be targeted, thus enabling more appropriate orbetter suited information or control signals to be provided with respectto the group of individuals, for example, taking into account one ormore features at a group level instead of at an individual level. Forexample, the method 100 according to various embodiments of the presentinvention is advantageously able to understand groupdynamics/characteristics and enables the provision of information thateffectively targets a group of individuals.

In various embodiments, the method 100 further comprises determining oneor more group-based features associated with the group of individualsbased on the one or more individual-based features determined for eachindividual in the group. In this regard, the group characteristicinformation may be determined based on the one or more group-basedfeatures determined for the group of individuals. Similarly, it will beunderstood by a person skilled in the art that the term “group-basedfeature” (which may also be interchangeably referred to as“group-associated feature”) may refer to any feature associated with agroup (i.e., at a group level or with respect to the group), such as acharacteristic or an attribute, of the group. It will be appreciated bya person skilled in the art that the presented invention is not limitedto any particular types of group-based features that can be determined,as long as the group-based feature(s) may be used for, or may facilitatein, categorizing or classifying purposes with respect to the group. Byway of examples only and without limitations, the group-basedfeatures(s) may include proxemics-based feature(s) or characteristic(s)(e.g., degree of separation or the interpersonal distances ofindividuals in the group, for example, whether they are within anintimate space, a personal space, a social space or a public space), ashape or configuration of the group (e.g., the shape or configurationformed by the group), sound/voice features (e.g., voice patterns in thegroup or detecting voice barge-in), group synchrony or behavior (e.g.,the synchrony of body movements of individuals in the group), grouporigin (e.g., the ethnicity of the majority of individuals in thegroup), and so on. It will be understood by a person skilled in the artthat each type of group-based features may be determined by one or morecorresponding classifier or analysis module based on variousindividual-based features determined. Furthermore, similarly asdescribed hereinbefore, it will be appreciated by a person skilled inthe art that various classification techniques/algorithms may be appliedfor producing a classification result/output based on one or more inputs(group-based features) obtained/detected, and the present invention islimited to any particular type of classification technique/algorithm.That is, it can be understood by a person skilled in the art that anyappropriate classification technique known in the art may be applied toproduce a classification result/output (group characteristicinformation) based on the one or more group-based features determinedfor the group of individuals. By way of example, SVMs are supervisedlearning models with associated learning algorithms that analyze datafor classification and regression analysis, and SMVs may build a modelbased on training data (e.g., training sets of one or a combination oftypes of features and the corresponding desired group characteristicinformation) for determining (e.g., predicting) the classificationresult/output based on inputs received (e.g., group-based featuresdetermined for the group of individuals).

In various embodiments, the step 102 of identifying the group ofindividuals comprises a step of determining a separation between eachadjacent pair of individuals (i.e., a separation between an adjacentpair of individuals for each adjacent pair of individuals) amongst theplurality of individuals; a step of determining an orientation of eachindividual of the plurality of individuals; and a step of determining,for each adjacent pair of individuals, whether to assign the adjacentpair of individuals as belonging to the group of individuals based onthe separation determined with respect to the adjacent pair ofindividuals and the orientation determined with respect to eachindividual of the adjacent pair of individuals. For example, theseparation may be derived from spatial features (e.g.,positions/locations, such as coordinates) of the individuals. In variousexample embodiments, a pairwise distance may be determined between eachadjacent pair of individuals and the separation between each adjacentpair of individuals may thus be indicated by or correspond to thepairwise distance determined.

In various embodiments, the step of determining whether to assign theadjacent pair of individuals comprises determining to assign theadjacent pair of individuals as belonging to the group on individuals ifthe separation between the adjacent pair of individuals determinedsatisfies a predetermined separation condition and the orientations ofthe adjacent pair of individuals determined satisfy a predeterminedorientation condition.

In various embodiments, the predetermined separation condition is thatthe separation determined is less than a predetermined separation (e.g.,a predetermined distance), and the predetermined orientation conditionis that the orientations determined are toward each other. For example,various studies in the past have been conducted and a number ofparticular interpersonal distances have been concluded to representvarious circumstances associated with an individual. For example, about0 to 0.45 m may represent an intimate space, about 0.45 m to about 1.2 mmay represent a personal space, about 1.2 m to about 3.6 m may representa social space, and about 3.6 m to 7.6 9m may represent a public space.Accordingly, the predetermined separation may be set as desired or asappropriate, for example, based on various studies conducted in thepast, such as but not limited to, about 3.6 m. In various embodiments,the orientations of a pair of individuals may be determined to be towardeach other as long as the direction vectors associated with the pair ofindividuals, respectively, are convergent, such as converging towards apoint or intersects at a point.

For example, a first adjacent pair of individuals may be determined tobelong as a group if they satisfy both the predetermined separationcondition and the predetermined orientation condition, and thus may beassigned to the same group (e.g., first group). In addition, a secondadjacent pair of individuals may also be determined to belong as a groupif they satisfy both the predetermined separation condition and thepredetermined orientation condition, and thus may be assigned to thesame group (e.g., second group). Furthermore, there may be an individualcommon to both the first and second groups, and in such a case, thefirst and second groups may then be merged as one group.

In various embodiments, the step 102 of identifying the group ofindividuals further comprises generating a schematic model for eachindividual of the plurality of individuals. In this regard, theseparation and the orientation are determined based on the schematicmodel of the respective individual.

In various embodiments, the schematic model is a skeleton model or aface model, and the separation is determined based on one or morepredetermined first portions of the respective skeleton model or facemodel and the orientation is determined based on a direction vectorgenerated based on one or more predetermined second portions of therespective skeleton model or face model. For example, in various exampleembodiments, in the case of a skeleton model, the first portion maycorrespond to the thorax portion of the skeleton model, and the secondportions may correspond to the left and right shoulder end portions ofthe skeleton model. For example, the direction vector may beperpendicular to a line (e.g., from a center of the line)joining/connecting the left and right shoulder end portions of theskeleton model. For example, in various example embodiments, in the caseof a face model, the first portion may correspond to a central portion(e.g., nose portion) of the face model, and the second portions maycorrespond to the left and right eye portions (or ear portions) of theface model. Similary, the direction vector may then be perpendicular toa line (e.g., from a center of the line) joining/connecting the left andright eye portions (or ear portions) of the face model. It will beappreciated to a person skilled in the art that various other types ofschematic models may be used to model or represent an individual asdesired or as appropriate, as long as the position and orientation ofthe individual can be derived from the schematic model.

In various embodiments, the step 104 of determining one or moreindividual-based features comprises determining each of the facialfeature(s), the body feature(s), the spatial feature(s), and the speechfeature(s) for each individual in the group of individuals. In thisregard, the step 106 of determining a group characteristic informationcomprises determining the group characteristic information based on eachof the facial feature(s), the body feature(s), the spatial feature(s),and the speech feature(s) associated with each individual in the groupof individuals determined.

FIG. 2 depicts a schematic block diagram of a system 200 for generatingan output with respect to a group of individuals according to variousembodiments of the present invention, such as corresponding to themethod 100 for generating an output with respect to a group ofindividuals as described hereinbefore according to various embodimentsof the present invention.

The system 200 comprises a memory 202, and at least one processor 204communicatively coupled to the memory 202 and configured to: identifythe group of individuals amongst a plurality of individuals in an areabeing monitored by one or more sensors; determine, for each individualin the group of individuals, one or more individual-based featuresassociated with the individual based on sensing data obtained from theone or more sensors; determine a group characteristic informationassociated with the group of individuals based on the one or moreindividual-based features determined for each individual in the group;and generate the output based on the group characteristic informationdetermined for the group of individuals.

It will be appreciated by a person skilled in the art that the at leastone processor 204 may be configured to perform the required functions oroperations through set(s) of instructions (e.g., software modules)executable by the at least one processor 204 to perform the requiredfunctions or operations. Accordingly, as shown in FIG. 2, the system 200may further comprise a group identifying module or circuit 206configured to identify the group of individuals amongst a plurality ofindividuals in an area being monitored by one or more sensors; anindividual-based feature determining module or circuit 208 configured todetermine, for each individual in the group of individuals, one or moreindividual-based features associated with the individual based onsensing data obtained from the one or more sensors; a groupcharacteristic determining module or circuit 210 configured to determinea group characteristic information associated with the group ofindividuals based on the one or more individual-based featuresdetermined for each individual in the group; and an output module 212configured to generate the output based on the group characteristicinformation determined for the group of individuals. For example, theoutput module 212 may generate an output in the form of a content, analert or an effect based on the group characteristic information, or inthe form of a signal or message comprising the group characteristicinformation for transmission to one or more devices or systemsconfigured for specific application(s), such that the one or moredevices or systems generate their output based on the signal or messagereceived.

In various embodiments, the system 200 corresponds to the method 100 asdescribed hereinbefore with reference to FIG. 1, therefore, variousfunctions or operations configured to be performed by the least oneprocessor 204 may correspond to various steps of the method 100described in hereinbefore, and thus need not be repeated with respect tothe system 200 for clarity and conciseness. In other words, variousembodiments described herein in context of the methods are analogouslyvalid for the respective systems or devices, and vice versa.

For example, in various embodiments, the memory 202 may further havestored therein a group identifying module 206, an individual-basedfeature determining module 208, a group characteristic determiningmodule 210, and/or an output module 212, as described herebefore withrespect to the method 100 according to various embodiments of thepresent invention, which are executable by the at least one processor204 to perform the corresponding functions/operations as described.

A computing system, a controller, a microcontroller or any other systemproviding a processing capability may be provided according to variousembodiments in the present disclosure. Such a system may be taken toinclude one or more processors and one or more computer-readable storagemediums. For example, the system 200 described hereinbefore may be adevice or a system including a processor (or controller) 204 and acomputer-readable storage medium (or memory) 202 which are for exampleused in various processing carried out therein as described herein. Amemory or computer-readable storage medium used in various embodimentsmay be a volatile memory, for example a DRAM (Dynamic Random AccessMemory) or a non-volatile memory, for example a PROM (Programmable ReadOnly Memory), an EPROM (Erasable PROM), EEPROM (Electrically ErasablePROM), or a flash memory, e.g., a floating gate memory, a chargetrapping memory, an MRAM (Magnetoresistive Random Access Memory) or aPCRAM (Phase Change Random Access Memory).

In various embodiments, a “circuit” may be understood as any kind of alogic implementing entity, which may be special purpose circuitry or aprocessor executing software stored in a memory, firmware, or anycombination thereof. Thus, in an embodiment, a “circuit” may be ahard-wired logic circuit or a programmable logic circuit such as aprogrammable processor, e.g., a microprocessor (e.g., a ComplexInstruction Set Computer (CISC) processor or a Reduced Instruction SetComputer (RISC) processor). A “circuit” may also be a processorexecuting software, e.g., any kind of computer program, e.g., a computerprogram using a virtual machine code, e.g., Java. Any other kind ofimplementation of the respective functions which will be described inmore detail below may also be understood as a “circuit” in accordancewith various alternative embodiments. Similarly, a “module” may be aportion of a system according to various embodiments in the presentinvention and may encompass a “circuit” as above, or may be understoodto be any kind of a logic-implementing entity therefrom.

Some portions of the present disclosure are explicitly or implicitlypresented in terms of algorithms and functional or symbolicrepresentations of operations on data within a computer memory. Thesealgorithmic descriptions and functional or symbolic representations arethe means used by those skilled in the data processing arts to conveymost effectively the substance of their work to others skilled in theart. An algorithm is here, and generally, conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities, suchas electrical, magnetic or optical signals capable of being stored,transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from thefollowing, it will be appreciated that throughout the presentspecification, discussions utilizing terms such as “identifying”,“determining”, “generating”, “computing”, “presenting”, “providing”, orthe like, refer to the actions and processes of a computer system, orsimilar electronic device, that manipulates and transforms datarepresented as physical quantities within the computer system into otherdata similarly represented as physical quantities within the computersystem or other information storage, transmission or display devices.

The present specification also discloses a system, a device or anapparatus for performing the operations/functions of the methodsdescribed herein. Such a system, device or apparatus may be speciallyconstructed for the required purposes, or may comprise a general purposecomputer or other device selectively activated or reconfigured by acomputer program stored in the computer. The algorithms presented hereinare not inherently related to any particular computer or otherapparatus. Various general-purpose machines may be used with computerprograms in accordance with the teachings herein. Alternatively, theconstruction of more specialized apparatus to perform the requiredmethod steps may be appropriate.

In addition, the present specification also at least implicitlydiscloses a computer program or software/functional module, in that itwould be apparent to the person skilled in the art that the individualsteps of the methods described herein may be put into effect by computercode. The computer program is not intended to be limited to anyparticular programming language and implementation thereof. It will beappreciated that a variety of programming languages and coding thereofmay be used to implement the teachings of the disclosure containedherein. Moreover, the computer program is not intended to be limited toany particular control flow. There are many other variants of thecomputer program, which can use different control flows withoutdeparting from the spirit or scope of the invention. It will beappreciated by a person skilled in the art that various modulesdescribed herein (e.g., a group identifying module 206, anindividual-based feature determining module 208, a group characteristicdetermining module 210, and/or an output module 212) may be softwaremodule(s) realized by computer program(s) or set(s) of instructionsexecutable by a computer processor to perform the required functions, ormay be hardware module(s) being functional hardware unit(s) designed toperform the required functions. It will also be appreciated that acombination of hardware and software modules may be implemented.

Furthermore, one or more of the steps of a computer program/module ormethod described herein may be performed in parallel rather thansequentially. Such a computer program may be stored on any computerreadable medium. The computer readable medium may include storagedevices such as magnetic or optical disks, memory chips, or otherstorage devices suitable for interfacing with a general purposecomputer. The computer program when loaded and executed on such ageneral-purpose computer effectively results in an apparatus thatimplements the steps of the methods described herein.

In various embodiments, there is provided a computer program product,embodied in one or more computer-readable storage mediums(non-transitory computer-readable storage medium), comprisinginstructions (e.g., a group identifying module 206, an individual-basedfeature determining module 208, a group characteristic determiningmodule 210, and/or an output module 212) executable by one or morecomputer processors to perform a method 100 for generating an outputwith respect to a group of individuals as described hereinbefore withreference to FIG. 1. Accordingly, various computer programs or modulesdescribed herein may be stored in a computer program product receivableby a system (e.g., a computer system or an electronic device) therein,such as the system 200 as shown in FIG. 2, for execution by at least oneprocessor 204 of the system 200 to perform the required or desiredfunctions.

The software or functional modules described herein may also beimplemented as hardware modules. More particularly, in the hardwaresense, a module is a functional hardware unit designed for use withother components or modules. For example, a module may be implementedusing discrete electronic components, or it can form a portion of anentire electronic circuit such as an Application Specific IntegratedCircuit (ASIC). Numerous other possibilities exist. Those skilled in theart will appreciate that the software or functional module(s) describedherein can also be implemented as a combination of hardware and softwaremodules.

In various embodiments, the system 200 may be realized by any computersystem (e.g., portable or desktop computer system), such as a computersystem 300 as schematically shown in FIG. 3 as an example only andwithout limitation. Various methods/steps or functional modules (e.g., agroup identifying module 206, an individual-based feature determiningmodule 208, a group characteristic determining module 210, and/or anoutput module 212) may be implemented as software, such as a computerprogram being executed within the computer system 300, and instructingthe computer system 300 (in particular, one or more processors therein)to conduct the methods/functions of various embodiments describedherein. The computer system 300 may comprise a computer module 302,input modules, such as a keyboard 304 and a mouse 306, and a pluralityof output devices such as a display 308, and a printer 310. The computermodule 302 may be connected to a computer network 312 via a suitabletransceiver device 314, to enable access to e.g. the Internet or othernetwork systems such as Local Area Network (LAN) or Wide Area Network(WAN). The computer module 302 in the example may include a processor318 for executing various instructions, a Random Access Memory (RAM) 320and a Read Only Memory (ROM) 322. The computer module 302 may alsoinclude a number of Input/Output (I/O) interfaces, for example I/Ointerface 324 to the display 308, and I/O interface 326 to the keyboard304. The components of the computer module 302 typically communicate viaan interconnected bus 328 and in a manner known to the person skilled inthe relevant art.

It will be appreciated by a person skilled in the art that theterminology used herein is for the purpose of describing variousembodiments only and is not intended to be limiting of the presentinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

In order that the present invention may be readily understood and putinto practical effect, various example embodiments of the presentinvention will be described hereinafter by way of examples only and notlimitations. It will be appreciated by a person skilled in the art thatthe present invention may, however, be embodied in various differentforms or configurations and should not be construed as limited to theexample embodiments set forth hereinafter. Rather, these exampleembodiments are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the present invention tothose skilled in the art.

In particular, for better understanding of the present invention andwithout limitation or loss of generality, various example embodiments ofthe present invention will now be described with respect to the outputbeing a targeted advertisement content. However, as mentionedhereinbefore, it will be appreciated by a person skilled in the art thatthe present invention is not limited to the output being a targetedadvertisement content, and the output generated may be for various otherapplications as desired or as appropriate, as long as the output isdesired to be dependent or based on the group characteristic informationassociated with a group of individuals determined, such as but notlimited to, a targeted security alert and an environment setting.

FIG. 4 depicts a schematic drawing illustrating an example overview of amethod for generating an output with respect to a group of individualsaccording to various example embodiments of the present invention. Asshown in the example of FIG. 4, there may be provided a system 400(e.g., corresponding to the system 200 as described hereinbeforeaccording to various embodiments) for generating an output with respectto a group of individuals. In the example, the system 400 may comprise,or may be communicatively coupled to, one or more sensors. The sensormay be any type of sensors as desired or as appropriate for obtaining orcollecting the desired data or information, such as but not limited to,a motion sensing device 406-1 (e.g., MICROSOFT KINECT or the like) forcollecting/generating motion data of individual(s), an image capturingdevice 406-2 (e.g., a camera) for capturing image data of individual(s),an audio capturing device 406-3 (e.g., a microphone) for capturing audiodata from individual(s), and/or a range sensing device 406-4 (e.g.,LIDAR sensor or the like) for generating distance data (i.e., pointcloud data) of individual(s). In various example embodiments, apositioning system (e.g., WiFi positioning) may be provided fordetermining the position/location of individual(s). It will beappreciated by a person skilled in the art that the different sensorsare not necessarily separate units and may be realized as one integratedunit as appropriate, such as an integrated unit with two or more ofmotion sensing, image capturing, distance (range) capturing and audiocapturing capabilities. Furthermore, it will be appreciated by a personskilled in the art that individual-based features may be derived basedon data (e.g., sensing data) obtained from one sensor or a combinationof different types of sensors (different types of modalities) as desiredor as appropriate. By way of example only, individual-based features maybe determined based on data obtained from a camera, a combination of acamera and a ToF sensor, a combination of a camera and a positioningsystem, and so on. For example, ToF sensor may provide betterlocalization, positioning and body orientation than a RGB camera, butthe type of sensors used may depend on various factors, such as thedegree of accuracy required and cost constraints.

The one or more sensors may be arranged to monitor individuals in adesignated area 410. The system 400 may be configured to (e.g., by agroup identifying module 206 therein) identify a group of individualsamongst a plurality of individuals 416-1 to 416-7 in the area 410 beingmonitored by the one or more sensors, such as based on sensing dataobtained from the one or more sensors. In the example, the system 400identifies a group 414 of individuals 416-1 to 416-5 as shown encircledin FIG. 4. It can also be observed that individuals 416-6 and 416-7 arenot identified as part of the group 414. In various example embodiments,individual 416-6 may be assigned to a second group by itself andindividual 416-7 may be assigned to a third group by itself. Forexample, in the case of the system 400 being an advertisement system,the system 400 may generate a targeted advertisement content based onthe group characteristic information determined for the group 414 ofindividuals, which may then be displayed on a display screen.

In various example embodiments, the targeted advertisement content (oran executable link to the targeted advertisement) may be sent to clientdevices (not shown) associated with the group 414 of individuals via anyappropriate wireless protocol known in the art, such as but not limitedto, a wireless push notification via Wi-Fi or Bluetooth. As a result,the group 414 of individuals may then be able to view the targetedadvertisement content on their respective client device.

FIG. 5 depicts a schematic drawing of an example skeleton model orrepresentation 504 generated for an individual at two example postures.It will be appreciated by a person skilled in the art that a skeletonmodel (e.g., as shown in FIG. 5 or any other suitable or appropriatetypes of skeleton model) for an individual may be determined orgenerated based on various conventional techniques known in the art,that is, suitable or appropriate conventional techniques thereof wouldbe understood by a person skilled in the art and thus need not bedescribed in detail herein. As shown, the skeleton model 504 has amultiple branching structure comprises a plurality of nodes at variouspoints corresponding to various joints/parts of a skeleton, each nodebeing an intermediate node or an end node, and a branch (arc) connectingappropriate/selected pairs of adjacent nodes.

For illustration purpose and without limitation, an example technique ofidentifying a group of individuals amongst a plurality of individualswill now be described according to various example embodiments of thepresent invention based on the skeleton model. The example technique mayinclude a first step of determining (e.g., estimating) a separation(e.g., pairwise distance) between each adjacent pair of individuals(based on the respective skeleton models generated) in the area (orscene) being monitored; a second step of determining (e.g., computing)an orientation of each individual (i.e., the orientation of thecorresponding skeleton model generated) in the area being monitored; anda third step of grouping individuals into one or more groups based on apredetermined separation condition and a predetermined orientationcondition.

In relation to the above-mentioned first step, for example, thethree-dimensional (3D) position of each skeleton model (e.g., Cartesiancoordinates of each node of the skeleton model) may be determined basedon a range sensor (e.g., a ToF sensor). It will be appreciated by aperson skilled in the art that any suitable node of the skeleton modelmay be chosen to represent the position of the skeleton model. By way ofan example, the node (node coordinate) corresponding to the thoraxposition of the skeleton model may be selected to represent the positionof the skeleton model, which may also be referred to as therepresentative node. Based on the representative node selected, thepairwise distance between a first individual (first skeleton model) anda second individual (second skeleton model) may then be determined, forexample, based on the following equation:

D=√{square root over ((X ₁ −X ₂)²+(Y ₁ −Y ₂)²+(Z ₁ −Z ₂)²)}  (Equation1)

where D is the pairwise distance, (X₁, Y₁, Z₁) is the Cartesiancoordinate of the first individual (first skeleton model), and (X₂, Y₂,Z₂) is the Cartesian coordinate of the second individual (secondskeleton model).

FIG. 6 depicts a schematic drawing of three individuals along with theirrespective orientation shown according to various example embodiments ofthe present invention. For example, in relation to the above-mentionedsecond step, determining the orientation (representative orientation) ofan individual may include locating a left shoulder node 604corresponding the left shoulder end of the skeleton model and a rightshoulder node 606 corresponding to the right shoulder end of theskeleton model, and generating a direction vector 608 perpendicular to aline 610 (e.g., from a center/middle point thereof) joining/connectingthe left shoulder node 604 and right shoulder node 606 of the skeletonmodel. The direction vector 608 determined may thus represent theorientation of the individual. In various example embodiments, thedirection vector 608 corresponds to the direction in which theindividual is facing (facing direction). It will be appreciated by aperson skilled in the art that the direction vector 608 may bedetermined or generated based on various conventional techniques knownin the art, that is, suitable or appropriate conventional techniquesthereof would be understood by a person skilled in the art and thus neednot be described in detail herein. For example, a conventional skeletontracker may provide a facing direction. As another example, a facingdirection may be determined based on a face detector of an imagecapturing device.

FIG. 7 depicts a schematic drawing of a top view of a plurality ofindividuals located in an area being monitored, along with theirrespective orientation shown, according to various example embodimentsof the present invention. In relation to the above-mentioned third step,grouping individuals into one or more groups may include grouping eachadjacent pair of individuals as one group if the pair of individualssatisfy both the predetermined separation condition and thepredetermined orientation condition. For example, as shown in FIG. 7,individuals 716-1 to 716-7 may initially be grouped into six groups720-1 to 720-6. For groups 720-1 to 720-4, each corresponding adjacentpair of individuals satisfy both the predetermined separation condition(e.g., their separation is within a predetermined separation) and thepredetermined orientation condition (e.g., their orientations areconsidered to be toward each other, e.g., converges towards a point),and thus the respective pair of individuals form a respective group. Onthe other hand, for example, individuals 716-7 and 716-1 do not satisfythe predetermined separation condition and individuals 716-6 and 716-2or 716-3 do not satisfy the predetermined orientation condition, andthus the respective pair of individuals do not form a respective group.In addition, initial groups 720-1 and 720-2 share a common individual,initial groups 720-2 and 720-3 share a common individual, and initialgroups 720-3 and 720-4 share a common individual (that is, initialgroups 720-1 to 720-4 are linked by at least one common individual), andthus, groups 720-1 and 720-4 may merge to form one group 722 ofindividuals 716-1 to 716-5. On the other hand, groups 720-5 and 720-6are each not merged with any other group(s) as they do not share anycommon individual with any other group. Accordingly, in the example, theplurality of individuals 716-1 to 716-7 are grouped into three separategroups 722, 720-5, and 720-6. As shown in FIG. 7, an isolated individual716-6 or 716-7 may also be considered as a group.

As mentioned hereinbefore, it will be appreciated to a person skilled inthe art that the present invention is not limited to a skeleton modelfor representing an individual and various other types of schematicmodels may be used as desired or as appropriate, as long as the positionand orientation of the individual can be derived from the schematicmodel. As another example, FIG. 8 depicts a schematic drawing of facemodels or representations 804, including eye portions (e.g., eye nodes),generated for three individuals in a group. It will be appreciated by aperson skilled in the art that a face model as appropriate or as desired(e.g., two-dimensional or three-dimensional) for an individual may bedetermined or generated based on various conventional techniques knownin the art, that is, suitable or appropriate conventional techniquesthereof would be understood by a person skilled in the art and thus neednot be described in detail herein.

Similarly, a group of individuals may be identified based on the facemodel according to various example embodiments of the present invention.The example technique may include a first step of determining (e.g.,estimating) a separation (e.g., pairwise distance) between each adjacentpair of individuals (based on the face models generated) in the area (orscene) being monitored; a second step of determining (e.g., computing)an orientation of each individual (i.e., the orientation of thecorresponding face mode generated) in the area being monitored; and athird step of grouping individuals into one or more groups based on apredetermined separation condition and a predetermined orientationcondition. Similarly, grouping individuals into one or more groups mayinclude grouping each adjacent pair of individuals as one group if thepair of individuals satisfy both the predetermined separation conditionand the predetermined orientation condition.

For example, as shown in FIG. 8, individuals 808-1 and 808-2 is groupedinto one group as they were found to satisfy both the predeterminedseparation condition (e.g., their separation is within a predeterminedseparation) and the predetermined orientation condition (e.g., theirorientations are considered to be toward each other, e.g., convergestowards a point). On the other hand, for example, individual 808-3 wasnot found to satisfy the predetermined separation condition with respectto individual 808-2 and was not found to satisfy the predeterminedorientation condition with respect to individual 808-1, and thusindividual 808-3 does not form a group with either individual 808-1 or808-2. In various example embodiments, to enhance visual effects, agroup box or frame 812 may be generated for enclosing or surroundingboth individuals 808-1 and 808-2 to indicate that they belong to agroup. Furthermore, in various example embodiments, a face box or frame814 may also be generated for enclosing or surrounding the respectiveface model whereby the size of the box may indicate a distance to areference point (e.g. the image capturing device). For example, asmaller face box enclosing a face model may indicate that thecorresponding individual is further away from a reference point than alarger face box. In the example of FIG. 8, both individuals 808-1 and808-2 may be at substantially the same distance away from a referencepoint and individual 808-3 may be further from the reference point thanindividuals 808-1 and 808-2.

FIG. 9 depicts a block diagram illustrate an example technique ofdetermining a group classification information associated with a groupof individuals based on a plurality of types of features determined forthe group according to various example embodiments of the presentinvention. As shown, a plurality of types of features (includingindividual-based and/or group based features) may be determined,including one or more body features 904, one or more facial features906, and one or more speech features 908. By way of examples only andwithout limitations, the one or more body features 904 may be determinedbased on a range sensor (e.g., ToF sensor) and may include, for example,a distance of an individual to the range sensor and/or a body movementsynchrony index associated with the group of individuals. In variousexample embodiments, a motion sensor may be provided to obtain motiondata of the individuals in the group, for example, for determining thebody movement synchrony index associated the group. The one or morefacial features 906 may be determined based on an image capturing device(e.g., a camera) and may include, for example, an emotional state of theindividual, the gender of the individual, the age of the individualand/or the ethnicity of the individual. By way of an example and withoutlimitation, an emotion estimating method based on facial expression isdescribed in U.S. patent application Ser. No. 15/288,866 titled “EmotionEstimating Method, Emotion Estimating Apparatus, and Recording MediumStoring Program” filed by Panasonic Intellectual Property Corporation ofAme on 7 Oct. 2016, the contents of which are hereby incorporated byreference in their entirety for all purposes. The one or more speechfeatures 908 may be determined based on an audio capturing device andmay include, for example, an emotional state of the individual, theethnicity of the individual, the speech synchrony index associated withthe group. It will be appreciated by a person skilled in the art thatthe present invention is not limited to the above-mentionedindividual-based or group-based features of an individual and othertypes of individual-based or group-based features may be determined asdesired or as appropriate. For example, possible types of body features904 may further include the proxemics (e.g., distance to the groupcenter x weight of the individual), the body openness index, thequantity of motion, and so on. Possible types of facial features 906 mayfurther include a gaze pattern (e.g., the number of time an individualis being looked at by other individuals in the group). Possible types ofspeech features 908 may further include loudness, speech duration, howformal or informal is the speech, personality analysis, and so on. Ascan be appreciated, there is generally no limit to the possible types ofindividual-based or group-based features that may be evaluated.

For example, with respect to the speech features 908, it will beappreciated by a person skilled in the art that the location of one ormore sources (and/or direction) of an audio/sound from one or moreindividuals detected may be determined based on various conventionalaudio source localization techniques known in the art, that is, suitableor appropriate conventional techniques thereof would be understood by aperson skilled in the art and thus need not be described in detailherein. As a result, for example, audio/sound from a particularindividual detected may be assigned to the particular individual bycomparing or matching the location (and/or direction) of the source ofthe audio/sound determined with the location (and/or direction) of theparticular individual determined. As a non-limiting example, for a soundfrom an individual, a linear function may be computed based on thedirection of the sound determined, and for each individual in the areabeing monitored, a corresponding linear function may also be computedthat passes through the individual (e.g., based on the position of thecorresponding skeleton model). Accordingly, the sound may then beassigned to the individual having an associated linear function which isclosest (e.g., best matches) the linear function of the sound.

It can also be appreciated by a person skilled in the art that each typeof individual-based or group based features desired may be determinedbased on any appropriate technique known in the art and thus need not bedescribed herein.

As shown in FIG. 9, after the plurality of types of features have beendetermined, a classifier 910 may then classify the group based on acombination of the plurality of types of features received and generatea classification result/output 912 as the group characteristicinformation (e.g., type of group) associated with the group.

By way of an example only and without limitation, a specific exampletechnique of classifying a group of individuals will now be described.The example technique includes a first step of determining (e.g.,estimating) the separation between each adjacent pair of individuals inthe group; a second step of determining a center position of the group;a third step of computing the emotional state of each individual in thegroup; a fourth step of determining the attire of each individual in thegroup; a fifth step of determining the gender, age, and ethnicity ofeach individual in the group; a sixth step of determining a bodysynchrony index associated with the group, for example, by computing theaverage skeleton position (e.g., average joint angle) and computing thedeviation for each individual; a seventh step of determining a speechsynchrony index associated with the group, for example, based on theintonation, the number of time individuals speak at the same time,whether individuals are taking turn to speak, and so on; and an eighthstep of classifying the group based on a combination of the types offeatures determined to generate a classification result/output as thegroup characteristic information associated with the group.

For illustration purpose only and without limitation, an example leaderdetection technique for detecting a leader in the group will now bedescribed according to an example embodiment of the present invention.For example, the example leader detection technique may be configuredtaking into account one or more (e.g., a weighted combination) of thefollowing factors: a leader may be the oldest in the group; a leader maybe gazed at by other individuals in the group more often; a leader maybe located close to a center of the group; a leader may be speaking moreoften; a leader may have an extraverted personality; and so on. Forexample, various movements such as head nods may also be useful cues indetecting a leader in a group. In the example, for each individual inthe group, a leader score may be determined based on the followingexample equation:

w₀*gender_(score)+w₁*age_(score)+w₂*proxemics_(score)+w₃*LOUdness_(score)+W₄*Extraversion_(score)+w₅*Speech_(score)   (Equation2)

where w₀ to W₅ are the respective weights assigned to each type offeatures for determining the leader in the group.

The leader may then be determined to be the individual in the group thatobtained the highest score based on Equation 2. It will be appreciatedby a person skilled in the art that the respective weights can be set asdesired or as appropriate based on various factors or circumstances(e.g., the location of the group, the country, the number of people inthe group, and so on). In various example embodiments, the weights canbe determined or learned using a linear regression model. In variousexample embodiments, the technique of detecting a leader may also beimplemented as a classification technique based on various machinelearning methods or algorithms known in the art.

By way of examples only and without limitations, FIGS. 10A to 10C depictthree schematic block diagrams of three example cases, respectively,illustrating example types of features of individuals in a group and theexample group characteristic information that may be determined for thegroup based on a classification of such example types of features. Forexample, the inputs to a classifier may include one or more of theposition (coordinates), age, gender, context, body orientation gazepattern, and an observation over time of individuals in the group. Forexample, the outputs to the classifier may include one or more of thetype (e.g., relationship) of group, an influencer (e.g., leader) of thegroup, and the group synchrony. For example, an individual analysis maybe determine one or more features such as age, gender, ethnicity,clothing, and emotion of the individual. For example, a group analysismay determine one or more features such as proxemics (e.g., who is closeto who), posture (e.g., who has closed/open posture), actionrecognition, head pose (who is looking at who), and voice patterns. Forexample, various group classification information may then bedetermined, such as the relationship of the group (e.g., couple,colleagues, father-child, friends, and so on), the influencer (e.g.,leader) in the group, the activity of the group (e.g., shopping, eating,travelling, and so on), and the type of content that may be advertisedto the group based on the context determined (e.g., restaurants,shopping, and so on).

By way of an example only and without limitation, FIG. 11 depicts anexample group classification technique configured to output a groupclassification information (e.g., friend, family, or colleague) based onvarious example types of features of the individuals in the group.

FIG. 12 depicts a block diagram illustrating an example system 1200 forgenerating an output with respect to a group of individuals according tovarious example embodiments of the present invention. As shown, theexample system 1200 comprises a plurality of sensors 1204 arranged forobtaining sensing data with respect to the plurality of individuals inan area being monitored, a plurality of types of classifiers or analysismodules 1208, each for determining the corresponding type of features ofeach individual in the group (individual-based features) and/or thecorresponding type of features associated with the group (group-basedfeatures), and a group characteristic classifier or analysis module 1212configured to generate a classification result/output corresponding tothe group characteristic information based on a fusion or combination ofthe different types of features determined. As shown in FIG. 12, forexample, the plurality of types of classifiers or analysis modules 1208may include a facial feature classifier configured to determine one ormore facial features based on the sensing data (e.g., image data)obtained from an image capturing device (e.g., camera), a body featureclassifier configured to determine one or more body features based onthe sensing data (e.g., image data and motion data) obtained from theimage capturing device and a motion sensing device (e.g., MICROSOFTKINECT or the like), a proxemics-based feature classifier configured todetermine one or more proxemics-based feature associated with the groupbased on the sensing data (e.g., image data) obtained from the imagecapturing device, and a speech feature classifier configured todetermine one or more speech features based the sensing data obtainedfrom an audio capturing device (e.g., microphone). As also shown in FIG.12, for example, the group characteristic classifier 1212 may be basedon SMV, PNN, or KNN, or a combination thereof. By way of example onlyand without limitation, the group characteristic information determinedby the group characteristic classifier 1212 may include one or more ofgroup relationship information (e.g., family, couple, colleagues, and soon), group leader information, a group activity information, socialdynamics information, group mood information (e.g., positive, negativeor neutral) and group objective information (e.g., shopping, leisure,dining, and so on).

Accordingly, various embodiments of the present invention areadvantageously able to generate an output (group characteristicinformation) with respect to the group of individuals identified (i.e.,at a group level). Therefore, various embodiments of the presentinvention are advantageously able to understand groupdynamics/characteristics and enables the provision of information thateffectively targets a group of individuals. For example, examples typesof group characteristic information may include one or more ofrelationship or context of the group (e.g., couple, colleagues,father-child, friends, and so on), leader of the group (e.g., theinfluencer), and activity of the group (e.g., shopping, eating,travelling, and so on). As a result, it is possible to determineappropriate content (e.g., better suited content that is more relevantto the group) to the group based on the relationship or context of thegroup determined.

As mentioned hereinbefore, the method and system described hereinaccording to various embodiments of the present invention may be appliedin a wide variety of applications. For example, possible types ofapplications may include targeted advertisement content (e.g., identifya group of tourist (including identifying the group leader based onvarious factors, such as holding a flag), and creating targetedadvertisement content for the group, e.g., based on the language spoken,age, gender, and so on of the majority of the individuals in the group),targeted security alert (e.g., scanning groups of people in crowdedplaces, such as trains, airports, and malls, bullying at school, and soon), entertainment (e.g., adaptive gaming in amusement parks), sports(e.g., predict outcome in a game based on players' dynamics), socialrobotics and computational attention, home electronics (e.g., smarttelevisions configured to recommend a program for a group ofindividuals), team management (e.g., detect tensions among team members(e.g., employees) such that appropriate actions may be taken to remedythe issue) and so on.

While embodiments of the invention have been particularly shown anddescribed with reference to specific embodiments, it should beunderstood by those skilled in the art that various changes in form anddetail may be made therein without departing from the spirit and scopeof the invention as defined by the appended claims. The scope of theinvention is thus indicated by the appended claims and all changes whichcome within the meaning and range of equivalency of the claims aretherefore intended to be embraced.

What is claimed is:
 1. A computer-implemented method for generating anoutput with respect to a group of individuals, the method comprising:identifying the group of individuals amongst a plurality of individualsin an area being monitored by one or more sensors; determining, for eachindividual in the group of individuals, one or more individual-basedfeatures associated with the individual based on sensing data obtainedfrom the one or more sensors; determining a group characteristicinformation associated with the group of individuals based on the one ormore individual-based features determined for each individual in thegroup; and generating the output based on the group characteristicinformation determined for the group of individuals.
 2. The methodaccording to claim 1, further comprising determining one or moregroup-based features associated with the group of individuals based onthe one or more individual-based features determined for each individualin the group, wherein the group characteristic information is determinedbased on the one or more group-based features determined for the groupof individuals.
 3. The method according to claim 1, wherein saididentifying the group of individuals comprises: determining a separationbetween each adjacent pair of individuals amongst the plurality ofindividuals; determining an orientation of each individual of theplurality of individuals; and determining, for each adjacent pair ofindividuals, whether to assign the adjacent pair of individuals asbelonging to the group of individuals based on the separation determinedwith respect to the adjacent pair of individuals and the orientationdetermined with respect to each individual of the adjacent pair ofindividuals.
 4. The method according to claim 3, wherein saididentifying the group of individuals further comprises generating aschematic model for each individual of the plurality of individuals,wherein the separation and the orientation are determined based on theschematic model of the respective individual.
 5. The method according toclaim 4, wherein the schematic model is a skeleton model or a facemodel, and said separation is determined based on one or morepredetermined first portions of the respective skeleton model or theface model and said orientation is determined based on a directionvector generated based on one or more predetermined second portions ofthe respective skeleton model or the face model.
 6. The method accordingto claim 3, wherein said determining whether to assign the adjacent pairof individuals comprises determining to assign the adjacent pair ofindividuals as belonging to the group on individuals if the separationbetween the adjacent pair of individuals determined satisfies apredetermined separation condition and the orientations of the adjacentpair of individuals determined satisfy a predetermined orientationcondition.
 7. The method according to claim 6, wherein the predeterminedseparation condition is that the separation determined is less than apredetermined separation, and the predetermined orientation condition isthat the orientations determined are toward each other.
 8. The methodaccording to claim 1, wherein the one or more sensors are selected fromthe group consisting of a motion sensing device, a range sensing device,an image capturing device, and an audio capturing device.
 9. The methodaccording to claim 8, wherein the one or more individual-based featuresdetermined are selected from the group consisting of a facial feature, abody feature, aspatial feature, and a speech feature.
 10. The methodaccording to claim 9, wherein said determining one or moreindividual-based features comprises determining each of the facialfeature, the body feature, the spatial feature, and the speech featurefor each individual in the group of individuals, and said determining agroup characteristic information comprises determining the groupcharacteristic information based on each of the facial feature, the bodyfeature, the spatial feature, and the speech feature associated witheach individual in the group of individuals.
 11. The method according toclaim 1, wherein the group characteristic information is a grouprelationship information indicating a relationship amongst theindividuals in the group.
 12. The method according to claim 1, whereinthe output is selected from the group consisting of a targetedadvertisement content, a targeted security alert, an environmentsetting, and a signal comprising the group characteristic information.13. A system for generating an output with respect to a group ofindividuals, the system comprising: a memory; at least one processorcommunicatively coupled to the memory and configured to: identify thegroup of individuals amongst a plurality of individuals in an area beingmonitored by one or more sensors; determine, for each individual in thegroup of individuals, one or more individual-based features associatedwith the individual based on sensing data obtained from the one or moresensors; determine a group characteristic information associated withthe group of individuals based on the one or more individual-basedfeatures determined for each individual in the group; and generate theoutput based on the group characteristic information determined for thegroup of individuals.
 14. The system according to claim 13, wherein theat least one processor is further configured to determine one or moregroup-based features associated with the group of individuals based onthe one or more individual-based features determined for each individualin the group, wherein the group characteristic information is determinedbased on the one or more group-based features determined for the groupof individuals.
 15. The system according to claim 13, wherein saididentify the group of individuals comprises: determining a separationbetween each adjacent pair of individuals amongst the plurality ofindividuals; determining an orientation of each individual of theplurality of individuals; and determining, for each adjacent pair ofindividuals, whether to assign the adjacent pair of individuals asbelonging to the group of individuals based on the separation determinedwith respect to the adjacent pair of individuals and the orientationdetermined with respect to each individual of the adjacent pair ofindividuals.
 16. The system according to claim 15, wherein said identifythe group of individuals further comprises generating a schematic modelfor each individual of the plurality of individuals, wherein theseparation and the orientation are determined based on the schematicmodel of the respective individual.
 17. The system according to claim16, wherein the schematic model is a skeleton model or a face model, andsaid separation is determined based on one or more predetermined firstportions of the respective skeleton model or the face model and saidorientation is determined based on a direction vector generated based onone or more predetermined second portions of the respective skeletonmodel or the face model.
 18. The system according to claim 15, whereinsaid determine whether to assign the adjacent pair of individualscomprises determining to assign the adjacent pair of individuals asbelonging to the group on individuals if the separation between theadjacent pair of individuals determined satisfies a predeterminedseparation condition and the orientations of the adjacent pair ofindividuals determined satisfy a predetermined orientation condition.19. The system according to claim 18, wherein the predeterminedseparation condition is that the separation determined is less than apredetermined separation, and the predetermined orientation condition isthat the orientations determined are toward each other.
 20. The systemaccording to claim 13, further comprising the one or more sensors,wherein the one or more sensors are selected from the group consistingof a motion sensing device, a range sensing device, an image capturingdevice, and an audio capturing device.
 21. The system according to claim20, wherein the one or more individual-based features determined areselected from the group consisting of a facial feature, a body feature,a spatial feature, and a speech feature.
 22. The system according toclaim 21, wherein said determine one or more individual-based featurescomprises determining each of the facial feature, the body feature, thespatial feature, and the speech feature for each individual in the groupof individuals, and said determine a group characteristic informationcomprises determining the group characteristic information based on eachof the facial feature, the body feature, the spatial feature, and thespeech feature associated with each individual in the group ofindividuals.
 23. The system according to claim 13, wherein the output isselected from the group consisting of a targeted advertisement content,a targeted security alert, an environment setting, and a signalcomprising the group characteristic information.
 24. A computer programproduct, embodied in one or more non-transitory computer-readablestorage mediums, comprising instructions executable by at least oneprocessor to perform a method for generating an output with respect to agroup of individuals, the method comprising: identifying the group ofindividuals amongst a plurality of individuals in an area beingmonitored by one or more sensors; determining, for each individual inthe group of individuals, one or more individual-based featuresassociated with the individual based on sensing data obtained from theone or more sensors; determining a group characteristic informationassociated with the group of individuals based on the one or moreindividual-based features determined for each individual in the group;and generating the output based on the group characteristic informationdetermined for the group of individuals.